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PREDICTION OF CHANGEOVER PERFORMANCE: 
OPERATIONAL TEST (OT) PARAMETERS FROM 
DEVELOPMENTAL TEST (DT) PARAMETERS 
VIA META-ANALYSIS 


Donald P. Gaver 
Patricia A. Jacobs 
Arthur Fries 


1. Introduction 

This paper sketches and examines some analytical statistical concepts and 
methodologies that should usefully inform and sharpen the process of 
military test and evaluation decisionmaking. The concepts generally fall into 
the broad category of combining information (CT) or meta-analysis. See Gaver 
et al. (1992) for examples and references. The term CI does not mean blindly 
simplistic and uncritical data pooling across either time or the different 
systems under evaluation. CI in test and evaluation would encourage and 
systematize quantitative descriptions and comparisons between systems' 
capabilities and limitations, over time and across comparable systems. It refers 
to explicit processes whereby judgments, experience and expertise, and data 
from previous and current military acquisitions, are systematically, 
transparently, and critically brought to bear on data-taking and analysis for 
either a particular current system acquisition, or on families of current and 


future projects. It quantifies aspects of corporate memory. 


The formalized CI process illustrated by the examples we provide has not 
yet proceeded far in practice in any organized way because useful historical 
data has not been identified, but requirements for more efficient T&E 


decisionmaking encourage the future development of such approaches. 


2. Combining Developmental and Operational Testing (DT and OT) 

Reliability Data 

Developmental Testing (DT) refers to the testing of a new or upgraded 
system in the course of its technical engineering development. In general, 
system DT is conducted by technical experts attentive to demonstration of its 
engineering performance requirements. Operational Testing .(OT) is 
conducted later, and by operational military personnel; the objective is to 
discover how the system is likely to behave in field operation and to uncover 
faults and obstacles to such operation. Because of the rigors of field operation 
there is the expectation that OT failure rates are likely to be higher than those 
prevailing in DT. We sketch analytical models that can represent such 


behavior in a quite economical or parsimonious way. 


Model I: A Fixed Changeover Effect Multiplier 

Suppose that 6; is the prior-to-changeover failure (or event) rate for 
system i, and @j is the corresponding post-changeover rate; i = I, 2, ..., J. 
Assume that before changeover system i fails in accordance with a Poisson 


process, so, over operating (exposure) time xj, system i fails dj dj =Q 1, 2, ..) 


d; 
; ee Sexe; (67%)? 
times with probability e 85%; Gea)" safter changeover that same system fails 
a : 


W; 

ae. ery: (Oi) * 

w; (wj =Q 1, 2, ...) times in operating time y; with probability e~?% (oxi) 
Wj: 


this independently for i=I, 2, ..., I; this is equivalent to assuming 





exponentially distributed times to/between system failures. It is assumed that 
the data initially available are (dj, xj, wi, yi, 1 =i, 2, ...). Our objective is to use 
these data to estimate any consistent change in rates (6;, @j) from prior- to 
post-changeover, and to use the estimated relationship to anticipate, and 
strengthen estimates of, the post-changeover rate of a new system. The 
different analysis approaches used here depend on different ways of 
characterizing an adjustment factor, k; kK is first taken to be a constant in 
Model I, applicable to all system changeovers. A subsequent setup, Model 11, 
allows the data to indicate the constancy of the relationship. 

Suppose there are I different systems for which both DT and OT data are 
available. Let Dj be the number of failures experienced by system i during 
developmental testing (DT) during an exposure time xj. Let Wj; be the 
number of failures experienced by that system during operational testing (OT) 
during exposure time y;. Model I assumes that {D;} are independent Poisson 
random variables with E[Dj] = 6;x; and {Wj} are independent Poisson random 
variables with E{Wj] = Kéjy;; that is, K is an unknown consfanfin this model. 
The log likelihood is, up to multiplication by irrelevant constants, 

I 
InL = &(6,x;data) = ¥' {(—5;x;) + dj 1n 5; - (x6;y;)+w;[Ink +1nd;}}. (2.1) 


i=l 


Setting #0, results in 
i 





(2.2) 


Setting - =0, 





K=——. (2.3) 


A recursive procedure to find the maximum likelihood estimates works; start 


with 5,(0) =d;/x;. The approximate variance of K can be obtained from Fisher 
information or by bootstrapping; details are omitted here, but see Gaver, 


Jacobs and Fries (1997). 


An important use of the estimate, K, is to project DT data for a new system 


into the post-changeover OT phase. Suppose, for instance, that we compute 





A d 
the isolated DT rate estimate for a new (the/ + 15*) system, 6743 = = +1 Then a 
I+1 
natural point estimate for the failure rate during OT could be 


141 = KOT41. 
Using the obvious independence and asymptotic likelihood approximations 


(Fisher information) the estimated standard error (se) of @y4; can be 


computed: 


Var(g) (de + vine =.) + Gear 


XJ+1\ 4141 XJ+1 X141 A*I+1 












This can in turn be used to assign approximate standard errors to predicted 
future OT performance, such as the probability that the future system will 


exhibit no/zero failures during a test or mission time x41): 


P{W = Osores Areal} =e Srrral™), 25) 





Numerical Examples 

Simulation was used to study the coverage properties of various 
confidence intervals for estimates of K in Modell. In each replication 20 
Poisson random numbers are generated having means 61%1, ..-, 61010, 
61ky1,---, 610Ky10, where K=4 and 6j, xi, yi appear in Table 1. This is a 


simulated version of raw observational data. 


TABLE 1 


System oi Xi Vi 
Number | DT Failure Rate DT Test Time OT Test Time 





(hours) (hours) 


1 20,000 5000 





2 0.0004 10,000 2500 
3 0.0006 | 666667. | 166667 
4 0.0008 | 5000 | 1250 





6 0.002 2000 500 
8 0.006 666.67 166.67 
9 0.008 500 | 125 
10 0.01 400 | 100 


For each replication 7 types of confidence intervals for K are calculated. 
The first uses the MLE estimate of kK and the asymptotic normal confidence 
limits with observed Fisher information. The next three procedures use 2000 
bootstrap replications where the bootstrap resampling is from Poissons with 
means dj, wj;. One bootstrap confidence interval procedure uses the 
percentiles of the bootstrap distribution. Another is a percentile-t procedure 
with the observed Fisher information of the bootstrap sample being used to 


estimate the standard error; (cf. DiCiccio and Efron (1996)). The third 





procedure uses the normal confidence interval procedure with the bootstrap 
standard error. The last three confidence intervals are also obtained by 
bootstrapping. However, in this case the 2000 bootstrap re-samples are drawn 
as follows. 

1. Obtain re-samples of pre-changeover (DT) data as random numbers from 

the Poisson distribution with mean b:x;. 

2. Obtain re-samples from post-changeover (OT) data as Poisson samples 
with mean K6;y; 
where é; and K are the original parameter estimates. 

The three confidence interval methods are the percentile, the percentile-t, 
and the normal confidence interval procedure with bootstrap standard error. 

Table 2 displays the results of the simulations. Displayed are the number 
of intervals that cover the true K = 4, and the mean and standard deviation of 
the width of the intervals. The results of the simulation do not differ by 
much for the different confidence interval procedures. Thus, for practical 
purposes the convenient asymptotic confidence interval seems adequate. 
Perhaps by luck the asymptotic interval not only covers as well as any, but is 
also shorter and less variable. 

Simulation is also used to study confidence intervals for the failure rate of 
the post-changeover OT failure rate that is projected from DT data for a new 
system, using kK. In each replication, data is simulated using the model with 
parameters in Table 1 with x = 4. In addition, the number of DT failures for an 
11th (the new) system is simulated by generating dj; from a Poisson 
distribution with mean 0.004 x 1,000 conditioned to be positive; that is, 614 = 
0.004 and x11 = 1000. The estimated DT failure rate of the 11th system bn = 





d31/x11 where d11 is the random number and @1;=*6,; where K is the 


estimate obtained from the data generated for the 10 systems. 


[Mean[SD[ | Mean[SD |__| Mean] SD. 
ee [> fees = fast = ae 
Normal 
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Bootstrap I D;(b) ~ Poisson (4;) W,(b) ~ Poisson (w;) 
Bootstrap II: D,;(b)~ Poisson (5:x;) W,(b) ~ Poisson (5:%y;) 





For each replication 5 types of confidence intervals for the new system’s 
OT failure rate, @11, are calculated. The first uses the MLE estimate of k, bi, 
and the asymptotic estimate of standard error and normal percentiles; this 
confidence interval is called the asymptotic normal interval. The next two 
confidence intervals use 2000 bootstrap replications, where the bootstrap 
resampling includes the additional random draw of d11(b) from a Poisson 
distribution with mean 411, the number of DT failures for the 11 system. 
The bth bootstrap estimate of 11 is @14(b) =K(b)d,1(b)/x11. One bootstrap 
confidence interval is constructed from the percentiles of the bootstrap 


distribution of @,;. The second procedure uses the normal confidence 


interval procedure with the mean and standard deviation of the bootstrap 
estimates. The fourth and fifth confidence intervals are also bootstrap 
confidence intervals but with the bootstrap samples for the first 10 systems 
being drawn as follows: dj(b) = Poisson mean 6x; and w;(b) ~ Poisson mean 
Kb;y;. The fourth confidence interval uses the percentiles of the bootstrap 
distribution of @11 and the fifth uses the mean and standard deviation of the 
bootstrap distribution and standard normal percentiles. The results appear in 
Table 3. Displayed are the number of intervals (out of the 100) that cover the 
true value of @11 = 0.016 and the mean and standard deviation of the width of 
the intervals. There is not much practical difference between the 5 confidence 


intervals procedures. 


WT % 
Coverage| Width | Coverage 
[SiMe SDT [Mean [SD |__| Mean] SD. 


Asymptotic 82 0.023 | 0.009 87 0.011 91 0.085 
Normal 


SE 


0003 


BootstrapI : D,(b)7 Poisson mean dj; W;(b) ~ Poisson mean w;, t= 1 10 











Bootstrap II : D,(b) — Poisson mean 6;x;;  W,(b) = Poisson mean 6;4y; i =1,...,10 








Model 11: A Variable/Adaptive Changeover Multiplier 

This next model generalizes the previous setup by allowing for possible 
variability in the DT-OT multiplier, x; we permit the training data sets, 
i=1,2,...,1, a chance to reveal their appropriate x-variability: each training 
data set is thought of as having its own xk-value, each a sample from a 
population with mean and variance to be estimated. If the variability of this 
population is sizable then the predictability (and usefulness) of the relation is 
questionable. We again assume {Dj} are independent Poisson random 
variables with mean 6;x;, and we let {Wj}, number of failures during OT, to 
be independent random variables. The conditional distribution of Wj, given 


Zj, is Poisson with mean @jy; = 6;Z;y; where Z; is conveniently taken to be a 


B 


ay and shape 


gamma-distributed random variable having mean p = E[Z;} = 


parameter B (its scale is a),so each system has its own individual DT-OT 
multiplier; these may be close to a mean value p but not necessarily tightly 
clustered around that mean. Note that this is not the same as a 
partially/Bayesian analysis of Model I with x an unknown constant described 
by a gamma prior. In the present model {Wj} turn out to be independent 
negative-binomial random variables that depend on the values of aand 8, 
which will be estimated from data. The value of /Var[Z;]/E[Z;] allows an 
analyst to get a rough idea of the cohesiveness of his data sets. 

The log-likelihood is, up to irrelevant constants, 

£=inL= 


I ‘ . (26 
| a +d;1n6; + if Ee + Blna+w; ind; -(B+w;)Inla+ ai a 


i=l 


Analysis of simulated data shows that procedures to maximize the full 


likelihood rather frequently misbehave: while a somewhat reasonable 


estimate of the gamma mean, [ = oe is usually obtained, the tendency is for 


the estimate of shape, £, to fly towards +, so the variance of the x-population 
tends to be badly underestimated. Such misbehavior of likelihoods has been 
previously noted when attempts are made to estimate one or more basic 
(“interest”) parameters in the presence of many other (nuisance” 
parameters (this may be nature’s way of telling an analyst to slow down); see 
Cox and Reid (1993), (1987). A partially Bayesian way of addressing the 
situation is to treat the unknown 6s as random and integrate them out 
(marginalize). Simulations are used to demonstrate that this method can tend 
to be reasonably reliable— much more so than is the full likelihood approach. 
A similar maneuver has been employed to estimate the common mean of a 
large number of different-variance normal populations, cf. Barndorff-Nielsen 
and Cox (1994). 


Assume {6;} are iid with a Jeffery’s prior. Then to marginalize on 6 carry 





out 
a; 
ee ae Cs 
P[D; = d;,W; = w,|Z; =x} = je Mi a oe BiOi¥i a 5 fi 
(2.7) 
d;+ i 
~ftal” 
Xi + KYj 
Put the mean p = B/«a. The integrated likelihood is proportional to 
L(t, P;data) = 
j (ey Prd (2.8) 





f B 2k UW; = nu set aes 
Ure y von Je si Bue om + us | a 


i=l 











After several algebraic steps, the log-likelihood is, up to irrelevant constants, 


I 


£(u, B; data) = > ou 2 ~InI(B)+ Blnx;— Biny; 


i=1 


J | Ce am 
+Ingexp|\H/“£u + (uj + B-Ynuf pyre 


Direct numerical integration has been used in what follows. 


(2.9) 


Numerical Examples 

Simulations were used to evaluate the above procedures. In each 
simulation replication the DT rates, 6j;,i =1, 2, ...,I were generated 
independently from a uniform distribution over [0, 10]. The times xj = yj = 1 
for all simulations. In the first two Cases, (A) and (B), the {Z;} values were 
generated from a gamma distribution having mean pp =4 and shape 
parameter B = 1.In Cases (C) and (D), the {Z;} were generated from a gamma 
distribution having mean p =4 and shape parameter B = 4. 

For each data set generated, the mean and the shape parameters of the 
gamma distribution are estimated using the integrated likelihood (2.9), and 
alternatively the method of moments. The numerical integration uses 
Simpson’s rule with up to 10th order difference correction for a step size k = 
0.01 (cf. Hamming (1973)) over [0,20] as implemented in A Graphical 
Statistical System, AGSS. 

In the present method of moments, the DT failure rate of the ith system is 
estimated by 5; =d;/x;. The multiple of the OT failure rate for the it system 
is estimated by K; =u; j biy;- The moment estimate of the mean of the gamma 


fim = 2 Dj and the moment estimate of the shape parameter of the 


gamma is 


a 


~ \2 
fy =a) _ (2.10) 


No attempt has been made to adjust for the (Poisson) variability of dj or w; in 
the above. The integrated log likelihood is searched until parameters change 
by less than 0.01. The search is started at the method of moments estimates. 

Table 4 displays the results of a simulation experiment. Each simulation 
has 10 replications. Each replication consists of I systems. The mean and shape 
parameters of the gamma distribution are estimated using moments and 
integrated likelihood. The table reports the mean (and standard deviation) of 
the 10 replications of the estimates of the mean and shape parameter of the 
gamma for the moment estimators and the integrated likelihood estimators 
for each case. 

The results displayed in Table 4 suggest the following. Both procedures 
estimate the mean of the gamma relatively well; they are both biased high, 
with the bias smaller for integrated likelihood. The results for B suggest that 
the method-of-moments estimate of the shape parameter can be biased on the 
low side for larger 8. The variance of the estimates suggests that the smaller 
the variance of the gamma distribution (larger 8, or shape), the more difficult 
it is for either of the procedures to accurately estimate the shape parameter B. 

The simulation results are not presented as at all exhaustive or definitive, 
but as suggestive of procedures that might well work in practice (integrated 
likelihood, but also simple moments), and others to be avoided (full 


likelihood). 
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TABLE4 
Estimates of Mean and Shape Parameter of the Gamma Distribution 
10 replications 


Cases\Methods Mean (Std Dev) of Mean (Std Dev) 
Estimates of Estimates 
Integrated Likelihood Moments 


Eu = 480 (1.75) 

Burt = 1.52 (0.50) 

Em = 433 (0.89) Him = 487 (1.32) 
Bur, = 1.66 (0.71) Bu = 1.22 (067) 
Huy, = 4.55 (1.47) Liu = 493 (1.82) 
Bur, = 5.38 (3.10) Byy = 1.81 (0.90) 
Eu =4:34 (0.90) iy = 4.81 (1.22) 
Burr, = 5.23 (4.28) Buy = 1.38 (0.80) 


(A) | w=4 B=1,1=10 


( 
( 


(C) |w=4 B=41=10 





(D) |u=4 B=41=20 








3. Likelihood-Based Pooling of Observations from Sensors with a 
Particular Range-Dependent Precision 


The next example describes the form of an estimate of a target item's range 
from observations by several co-located sensors with range-dependent 
precision. Suppose there are s (s 2 1) sensors capable of detection and range 


determination of targets at various ranges. Here is a model: let 


Rj =range estimate of ih sensor, i=I,2 ...,§ 
if r is the true range of target, then suppose that all Rj are 
independent and normal/Gaussian, with 
E[R;|=r 
(3.1) 
Var[R;]= 0717. 
The objective is to estimate r, using all information available in the above, 


which means use the fact that the variance also depends on the mean. 


13 


Treating r as an unknown parameter one can write down its likelihood, 


given observations 


Ry =x, Ro =x, ..., Re = Xz. 


Itis 
-1( — Po??? 
8 xXj7T 
@2\" : 
IN; X1,X97.--,X5) = | —_——_ (3.2) 
( ss ~ s) it V2 Oj 


The log-likelihood is 


2 isi OjT 
(3.3) 


25 1 
or, if @=1/r 
$ (x,0-1) 
£(0;data) = oy (% 7 ) +sln@. (3.4) 
2 i=1 1 
Now 
By ee 8 
oe 254 oF 7] 
5 (3.5) 
ee ae ae 
= -0> =i + Y+ 
i=l oF i=l oF 0 


so setting this equal to zero yields a quadratic equation for @ 


2 
62] 21 |_| S / |~s=0. 
bey | 4) 


q 


The acceptable solution is 
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, (3.6) 


(3.7) 





Under certain circumstances (e.g. oj = O(O.1)) the second term inside the 
radical will be much smaller than the first; neglecting it gives the weighted 


estimate 


Me 


d(tlet) dxt/(ot?) 
(s/o?) Ee (ot?) 


It is surprising that, while the rightmost formula weights as the inverse of the 
estimated/observed-range-calibrated variance, i.e. (07?) 5» a natural 
2,2\ 

rye 


mde 
i 
—" 


(3.8) 


“™> 
i 


Me 


i=1 


me 
{| 


surrogate for V(6; then, instead of weighting the raw observations xj, it 


weights their fourth power with the normalizing factor necessarily involving 
weighted third powers. The accuracy of these estimates should be compared to 


the simple linearly-weighted estimate that recognizes true range dependence: 


¥ x(/o?r ?) ¥x,/0? 


= i=1___. (3.9) 


¥ (1/021?) Dy/o? 


i=] 


Fp = i 
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where the unknown true range dependence that influences the variance 
cancels in this (normal) dispersion model! This is independent of the form of 
the range dependence, g(r) (here g(r) = r2) provided it is the same for all 
sensors. Numerical investigation shows that the linear estimate (3.9) 
performs nearly optimally, and is certainly highly convenient in applications. 
Numerical Illustration 

We have simulated the results of observing a target by five different 
sensors and then combining the results. The situation and results appear in 
Table 5, and the figure provides further insights. 

As expected the mle approach (3.7) generates estimates that closely 
concentrate around the true mean (r= 1000); the standard error of these 
estimates is smaller than those of the other two. The approximation (3.8) is a 
bit high on the average (dropping a positive term in the denominator is the 
reason). The properties of the linear calculation (3.9) are gratifyingly similar to 
those of the mle; ease of calculation is welcome. 


TABLE 5 
Estimates of Range Determination by Combination 
500 replications 


r= 1000; of =03 =0.09; 0f =07 =0% = 0.05 








Eq. Number (3.7) (3.8) (3.9) 
Mean 1006.4 1068.8 991.2 


| Std. Dev. of Estimates| 134.6 | 143.1 | 141.1 | 





Mean Sq. Error | 181186 | 251692 199399 


4. Alternatives to the Circular Normal Dispersion Model: ''Robust CEP” 
Experience shows that in projectile (e.g. missile, gunfire) testing it is often 
the case that some individual shots deviate from aim point more wildly than 


described by the customary circular normal model. One way of providing a 





model for this to guide data analyses and perform CI across the DT to OT 


changeover is to stochastically mix: letting p =1/ o* bea precision parameter, 
think of it as being chosen randomly, perhaps (not necessarily) from shot to 
shot, and then presume that the random p has a distribution and use it to 
remove the condition on precision. 


I we start with standard circular normal dispersion, conditional on p, 
2 -irp 
PIR> rp= (1/0 =p}=e 2 (4.1) 
then when the condition is removed 
1 ope 1? \x 
P{R>r}= He | Je Gr) dF,(x). 
0 


Gamma Variation of the Precision Parameter 
Let the variability in the precision, p, be described by the gamma (af) 


density 
~OX (cx)? 
dF,(x) = fo(x;a, B)dx =e ——~— oud; (4.2) 


for which E[p] = B/ a= p, Varlp] = B/ a2. 
Then 


a 


=| “ 1 71 - 
rsore-l or) - [cape “(*2P8) ° 


The CEP satisfies for this distribution (which has a long or fat ’’Pareto tail’’), 


2 -B 
pp-~1| _1 
: + 5 p | a (4.4) 


This gives 


(4.5) 





The CEP approaches J2in2 //p = 1.177 as B > ~ if p =1, in which case there 
is no shot to shot variation in precision. If we maintain the mean of the 
dispersion distribution at p = 1 and reduce B (increase variance) B = 1 (and 
P =1) we find rcgp = 2 = 1.414, a 20% increase; as B decreases further the CEP 
increases indefinitely, induced by the great shot-to-shot variability. 
Parameter Estimation by Likelihood 

Maximum likelihood estimation of the parameters of the miss-distance 
distribution (4.3)can be easily done when all miss distances of a series of n 
shots are recorded: 71,12, .., ty. That is, none have been (prematurely) deleted 


as outliers. The density function of a miss distance is 


2 1 (B+1) 
fr(rio, B)= ‘ +53 #} rp; (4.6) 
the log-likelihood function is 


£(a, B;r)=-(B+ DIE + re F) +ninp. (4.7) 


To maximize, differentiate and set the derivatives equal to zero; solve. The 


equations appear as 


n 2 
a __ (B+) i. |4Z=9 (4.8,a) 
dp 2B j\1+1 p/2B) P 
or 
n 2 
i. B+1 ay. zi (4.8,b) 
Pp 2B ng1+r? p/2B 





and 





a | ( | (B+) SI). 1 
—=-S tn 147 + —4+|=0. (4.9,a) 
ap 2 2p? 2 les 


Because of (4.8,b) the second term allows 


pant Z\-4 B (4.9,b) 


An iterative solution of (4.8,b) and (4.9,b) is promising: start from B(0)=1, 
p(0) = 2B( a2 B(O) _ 1 / (rmep)- where ryep = median of the ordered miss 


distances. Use of Fisher information or bootstrapping will furnish standard 
errors of estimates. 
Modeling DT-OT Miss-Distance Data Combination 

A convenient approach to represent the difference between DT and OT 
miss distances in the current context is to utilize a proportional hazard or 
Lehmann alternative device. If Rp(Ry) are respectively location errors or 


shot shot deviations for DT and OT we put in (4.3) for the DT dispersion 


P{Ry > rae, B} = (P{Rp > ra, By 


Bx (4.10) 


=| t4(r°/2)p 46 


The corresponding CEP turns out to be 


TCEp = lap (atx =e (4.11) 


so here « turns out to be small if there is pronounced degradation of precision 


in OT over that in DT. 
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The density of OT miss distances is seen to be 


- 1 
Riep wane a1 oe” (4.12) 
2°58 ipa 


DT-OT Data, and Parameter Estimation 


Suppose we have DT miss-distance data from system i i =1, 2,..., I, 


denoted by r(DT) = Ujj,j =1,2, .., i, and corresponding OT miss distances, 
12(OT) = Vij, J = 1,2, ...,mj. Under Model I assumptions, i.e. a fixed 
(unknown) «x, the likelihood is 

Ion; 1 -(Bj+1) m on. 1 -(Bjx+1) 
Lp, pe ata)=\) lise B; aL Ti+ D; z| DiK (4.13) 


dropping factors independent of parameters. Hence the log-likelihood is 


I 
4(p, B,«;data) = $)- (B; +1 eal tn). (Bik +1 Sem +255) 





(4.14) 
Sing; + ( Inp; + Ink) |. 
j=l k=1 
From this, 
nj; = ; mM; 
SE so BON a ES at 
Pi Bi 1+ tpyp = Pe 1+ D/A OP 
or 
; nj us : mij ; 
1__ 1 | +1 aoe (Bj +1) — Pik __ (4.15,b) 
pi nj+m;| 28; j=l 1+ “45/8; 2Bi eat 14+ P;/B; 











(4.16,a) 


m; ™m; 5 
«(1+ kA 2 5 | eS 5 p; =9 
(et No Be} 2pF 7514+ +p; 4 


Because of (4.15,b) the second term allows 


1 
a In| 142 Bi l i vik | (4.16,b) 
B; “ald (1+ +e SIn es 2 B; 





j=l 
of I mj Dy — 1 : 
EON a BS ne oe oe ls 2. 4.17 
Se 2 | 7 nz] i: (4.17) 
Thus, 

I m; v 

6:14 ik Bi) 

i=l __k=1 i 


(4.18) 


An iterative solution of (4.15,b), (4.16,b) and (4.18) is promising: start from 
7 2 
B;(0) = 1,p;(0) = 2/ [median( ny --+%in and then solve for K(0) using (4.18). 


Notice that a Model II version of the present setup can be explicitly carried 
out: if the mixing distribution is gamma, as before, all necessary integrals 
come out in closed form and the analysis carried forward. This work must be 
postponed for the present. 

Numerical Example 

Suppose there is historical data on 5 similar systems. Each system has 20 

observations during DT, and 20 observations during OT. Odd-numbered 


systems have (f, P) = (0.5, 1). Even-numbered systems have (8, p) = (1, 2). 
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The common xk = 0.8. Figures 2 - 4 present results from 500 replications of a 
simulation: “data” were sampled from the above.model and the parameters 
estimated. Figure 2 presents histograms for the estimate of (8, p) for the odd 
numbered systems, and Figure 3 presents histograms for estimates of (8, p) for 
the even numbered systems. All estimates result from the iterative schemes 
of (4.15,a)- (4.18). Figure 4 presents a histogram of estimates of K The 
estimates appear well-behaved in that their histograms cluster well around 


the true (here known) values, doing so in an appropriately normal fashion. 


5. Discussion 

The present paper examines a selection of problem types typical of the 
testing environment. Emphasis is placed on the issue of borrowing 
information from the DT period to strengthen decisions concerning OT; this 
can be useful to inform the decision maker of the advisability of the 
immediate initiation of OT. Some attention is also given to auxiliary 
information, i.e. with respect to shot-to-shot variability change with range, 
and the occurrence of ”fat-tailed” outlier-prone shot dispersion distributions. 


We plan to further address such problems in future. 


ee 
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Figure 1 
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